In the transmission of a video signal over a fixed channel, the amount of data that must be coded and transmitted may vary for each video frame of a video sequence. Therefore, the coding and transmission method must be capable of handling variable amounts of data. Transmission of variable bit rate video over a fixed rate channel is an important problem for any codec which transmits variable length codes over the channel. Most of the existing standards in video belong to this class of coders.
Transmission of a variable rate bitstream over a fixed rate channel requires a buffer to regulate the bitstream being generated by the quantization process. The buffer must have a capacity that is large enough to handle at least one frame. Because the amount of data per frame may vary, the buffer must be large enough to handle the frame having the largest amount of data. A rate control mechanism is used to prevent the buffer from underflowing or overflowing by maintaining the output bitstream within a limited range. This may be achieved by controlling the quantization step size used in the coding loop. A larger quantization step size generally corresponds to a coarser quantization and smaller bit rate at the output. A smaller quantization step size generally corresponds to finer quantization and a larger bit rate at the output. Each time the quantization step size within a frame is changed, additional bits must be sent to the decoder to inform the decoder of the change. By changing the quantization step size, the number of bits required in coding a particular frame of a video sequence can be adjusted. The goal of most rate control methods is to minimize fluctuations in the bit rate, and thereby stabilize output quality, by maintaining a constant distortion over different blocks of a frame or different frames of a given sequence.
Until recently, most work on rate control has concentrated on higher bit rates like MPEG. (See, for example, A. Ortega and K. Ramchandran, "Forward adaptive quantization with optimal overhead cost for image and video coding with applications to MPEG video coders," Proceedings of IS&T/SPIE, Digital Video Compression, San Jose, Calif., February 1995; W. Ding and B. Kiu, "Rate-Quantization Modeling for Rate Control of MPEG Video Coding and Recording," Proceedings of IS&T/SPIE, Digital Video Compression, San Jose, Calif., February 1995). More recently, attempts have been made to extend this problem to lower bit rates (See, for example, A. Bist and P. Sriram, "A Perceptually Motivated Constrained Trellis Based Rate Control Scheme for TMN Video Codec," Rockwell Signal Processing Conference, Thousand Oaks, Calif., April 1995; T. Weignand, M. Lightstone, T. G. Campbell and S. K. Mitra, "Rate-Distortion Optimized Mode Selection for Very Low Bit Rate Video Coding and the Emerging H.263 Standard," pre-print, July 1995).
Several approaches have been proposed to attack the problem of bit rate control. One method considers the problem as a feedback control problem. Another approach views the problem of bit rate control as a constrained optimization problem. This approach has been used to assign bits among different frames in MPEG encoding. A third approach attempts to model the distortion-rate functions of the video sequences and use these models in assigning bits among different frames (See, for example, A. Puri and R. Aravind, "Motion Compensated Video Coding with Adaptive Perceptual Quantization," IEEE Transactions on Circuits and Systems for Video Technology, Vol. 1, No. 2, pp. 351-3361, December 1991).
As transmission bit rates decrease and buffer size remains limited, the need for a good rate control method for video becomes more acute. Standard linear rate control methods suffer from low performance and frame jerkiness. Most known methods are simplistic and fail to provide efficient distribution of bits within portions of one frame and between different frames.
It is an object of the present invention to provide an efficient and judicious method and apparatus for compressing, transmitting and decoding large amounts of video image data for frames of a moving image, and for doing so over a telephone line having a limited bandwidth and in accordance with International Telecommunications Union (ITU) standards for video compression algorithms, including H.261 (for ISDN lines) and H.263 (for telephone lines).